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Abstract 

Teams of Unmanned Aerial Vehicles (UAVs) form typical networked cyber- 
physical systems that involve the interaction of discrete logic and continuous 
dynamics. This paper presents a hybrid supervisory control framework for 
the three-dimensional leader follower formation control of unmanned heli- 
copters. The proposed hybrid control framework captures internal interac- 
tions between the decision making unit and the path planner continuous 
dynamics of the system, and hence improves the system's overall reliabil- 
ity. To design such a hybrid controller, a spherical abstraction of the state 
space is proposed as a new method of abstraction. Utilizing the properties 
of multi-affine functions over the partitioned space leads to a finite state 
Discrete Event System (DES) model, which is shown to be bisimilar to the 
original continuous-variable dynamical system. Then, in the discrete domain, 
a logic supervisor is modularly designed for the abstracted model. Due to the 
bisimilarity between the abstracted DES model and the original UAV dynam- 
ics, the designed logic supervisor can be implemented as a hybrid controller 
through an interface layer. This supervisor drives the UAV dynamics to sat- 
isfy the design requirements. In other words, the hybrid controller is able to 
bring the UAVs to the desired formation starting from any initial state inside 
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the control horizon and then, maintain the formation. Moreover, a colhsion 
avoidance mechanism is embedded in the designed supervisor. Finally, the 
algorithm has been verified by a hardware-in-the-loop simulation platform, 
which is developed for unmanned helicopters. The presented results show 
the effectiveness of the algorithm. 
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1. Introduction 

Formation control of the Unmanned Aerial Vehicles (UAVs) is a typical 
cooperative control scenario in which a group of UAVs jointly moves with 
fixed relative distances j2[, jsf). Formation control can leverage limited 
capabilities of individual UAVs so that as a team, they are able to per- 
form complicated tasks such as mapping, search, coverage, surveillance, and 
transportation, cooperatively. They can also mutually support each other in 
a hostile or hazardous environment ([l|, jij, jsf). 

In general, a formation scenario consists of several subtasks. Firstly, the 
UAVs should be controlled to form the desired formation. After reaching 
the formation, the UAVs should maintain the achieved formation while as a 
team, they have to track a desired path. Moreover, to guarantee the mission 
safety, the control algorithm is required to take care of inter-collision between 
the vehicles. 

To address these problems, different methods have been developed in the 
literature. For reaching the formation, there are several existing methods 
such as optimal control techniques, navigation function, and potential field 
(01, (sf, jsj , Maintaining the formation can be seen as a standard control 
problem in which the system's actu al p osition has slightly deviated from the 



desired position (|8|, |9|, [lO|, (jj). Finally, in (l3|, [l5|, [l6|, and 



17| , different mechanisms for collision avoidance have been introduced using 
predictive control, probabilistic methods, MILP programming, invariant sets, 
and behavioral control. 

To obtain a complete solution for the formation control, one way is to 
separately design a controller for each of these tasks in a decoupled way, and 
then, a decision making unit is needed to orchestrate the switching between 
these subcontrollers according to different scenarios. Although the negligence 
of the interaction between the continuous dynamics and the discrete logic de- 
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cisions simplifies the design, this may degrade the rehabihty of the overall 



system, and may cause unexpected failures as it happened in Arian 5 [18 
Hence, a more comprehensive analysis requires an in-depth understanding of 
the interplay between the continuous dynamics and the discrete supervisory 
logic of the system. One suitable solution for this problem is to utilize the 
hybrid modelling and control theory (jl9|, 20 1, 2l||) that provides a mathe- 



matic framework to capture both the continuous and the discrete dynamics 
of the system, simultaneously. 

In this paper, we propose a 3-D hybrid supervisory control architecture 
for the path planner layer of the UAV helicopters that are involved in a 
leader-follower formation mission. Furthermore, a new method of abstrac- 
tion is introduced, which uses the spherical partitioning of the state space. 
Utilizing multi-affine functions over the partitioned space, the designer is able 
to make each partition as an invariant set or to deterministically drive the 
UAV towards one of its adjacent partitions. The proposed abstracted model 
of the system can be captured by a finite state machine, which can be fur- 
ther studied through the Discrete Event Systems (DES) supervisory control 
theory initiated by Ramadge and Wonhom 2^. Within the DES framework, 
we can modularly design the discrete supervisors for reaching the formation, 
keeping the formation, and avoiding the collision. 

To implement the designed discrete supervisor, inspiring from [19J], an 
interface layer has been constructed to link the discrete supervisor and the 
continuous plant. It can be shown that the discrete model of the plant and 
its partitioned model have a bisimulation relation, so that they can exhibit 
the same behavior. 

The proposed method is based on spherical partitioning of the state space 
using the properties of multi-affine functions. Indeed, multi-affine functions 
are a large class of systems that are decidable under triangulization or rectan- 



gulization of the state space ((23|, [2^, |25|). So far, these methods have not 



been used in the UAV path planning and formation control. Moreover, the 
direct path towards the desired point is not available in a react angulized or 



trangulized space. In [26|, we proposed a 2-D formation algorithm based on 
polar abstraction of the state space for which the direct path towards the tar- 
get is applicable, which facilitates the implementation and the design of the 
formation algorithm. However, in ^], it had been assumed that the UAV's 
altitude remain unchanged and irrelevant. Compared with [26], the main 
contributions in this paper are that firstly, the results are extended to the 
3-D space and the spherical partitioning of the state space is provided. The 
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extension of polar abstraction to the spherical abstraction is not trivial as the 
structure of sectors in polar and spherical coordinate systems are different. 
Moreover, the resulting DES models are different. Secondly, to show that 
the designed controller for the abstract model works for the original plant, 
the bisimulation of the partitioned model of the plant and its abstracted 
DES model is proved and the bisimulation relation is accurately described. 
Thirdly, the algorithm has been verified through the hardware-in-the-loop 
simulation platform developed by our research group. 

The rest of this paper is organized as follows. First, the problem of 
formation control is formulated in Section |2l and then. Section |3] describes 
the principles of the spherical partitioning of the state space. Section H] 
utilizes the properties of multi-affine functions over the partitioned state 
space. In Section |5], two important control features, the invariant region and 
the exit facet, are introduced. Section |6] abstracts the partitioned model 
into a finite state machine and proves the bisimulation relation between the 
partitioned model of the plant and its abstract model. In Section [TJ the 
DES model of the system has been developed and a supervisor has been 
modularly designed, to handle reaching the formation, keeping the formation, 
and colhsion avoidance, accordingly. Section |8] describes how the discrete 
supervisor can be applied to the plant and how the closed-loop system works. 
Section [9] demonstrates the hardware-in-the-loop simulation results to verify 
the proposed algorithm. The paper is concluded in Section [101 



2. Problem formulation 



The modelling and low-level control structure of the NUS UAV helicopter 
is explained in 27| and jisj. This UAV is a Raptor-90 helicopter with 1410 



mm the full length and 190 mm full width of fuselage j29|. For this helicopter 



we have used a hierarchical control structure whose inner-loop controller 
stabilizes the system using ifoo control design techniques and its outer-loop 
is used to drive the UAV towards the desired position (Fig. [1]). The inner- 



loop is fast enough to track the given references |28|, so that the outer loop 



dynamics can be approximately described as follows: 



X = u xeR^, ueU CR^, (1) 

where x is the position of the UAV; u is the UAV velocity reference 
generated by the formation algorithm, and U is the velocity constraint set, 
which is a convex set. 
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Figure 1: Control Structure of the UAV. 



Now, consider the follower velocity in the following form: 

y follower ~ ^leader ~l" ^ei- (2) 

Having the velocity and position information of the leader, the follower 
UAV should reach and keep the formation by tuning the relative velocity, 
Vrei- Alternatively, one can consider a relatively fixed frame, in which the 
follower moves with the velocity of Vrei and the leader has a relatively fixed 
position. The problem is to design a supervisor in the outer-loop to bring the 
follower UAV to the desired distance with respect to the leader position to 
form the expected formation. Apparently, the desired position moves when 
the leader changes its position. Here, the control horizon is the sphere, 5"^^, 
with the radius of Rm that is centered at the desired position. The formation 
problem can be stated as follows: 

Problem 1. Given the dynamics of the path planner as (QP and the velocity 
of the follower in the form of design the formation controller to generate 
the relative velocity of the follower, Vrei, such that starting from any initial 
state inside the control horizon, it eventually reaches the desired position, 
while avoiding the inter- collision between the leader and the follower. More- 
over, after reaching the formation, the follower UAV should remain at the 
desired position for ever. 

To address this problem, we will propose a new method of abstraction 
based on spherical partitioning and then, as the follower's outer-loop dy- 
namics (II]) is a multi-affine function, we will deploy the properties of these 
classes of functions over the partitioned space. The resulting partitioned sys- 
tem can be bisimilarly captured by a finite DES model for which there are 
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(a) (b) 

Figure 2: (a) Spherical coordinate system and (b) A partitioned sphere Sr^^. 

well-established formal analysis and synthesis tools within DES supervisory 



control framework 30 , as described in the next section. 



3. Spherical partitioning 

Consider a plant with continuous dynamics of a; = g{x) = h{x,u{x)), 
defined over the sphere Sr^, with the radius of Rm, where u{x) is the control 
value computed based on feedbacked position of the system. The sphere Sr^ 
can be partitioned in the spherical coordinate system with r>0,0<^<27r, 



and < (j) < TT (Fig. 2(a)). The curves {r = ri\0 < < R^, fori < j : 
ri < rj, i,j = l,...,nr.,ri = 0, r^, = Rm}, {0 = 9i\0 < 9i < 2n, fori < j : 
di < 6j , i,j = 1, ...,ne, 6i = 0, 6ng = 2tt}, and {0 = 0i | < 0i < vr, fori < 
j : (pi < (pj , i,j = 1, n0, , 01 = 0, = it}, with n^, ng,n^ > 2, partition 



the control horizon Sr^. Equivalently partitioning (Fig. 2(b)), we will use 
{r^ = ^(i-1), i = l,...,nr}, {Oj = ^(j-1), J = l,...,ng}, and {0^ = 
-^zPiik ~ 1)? k = l,...,nA as partitioning curves. Using this notation, we 
will have (n,. — l){ng — 1) (n^ — 1) regions. A region Rij^k = {x = (r, 6, 0) | < 
^ < f^i+i, Oj < 9 < Oj^i, 0A: < < 0fc+i} is a subset of Sr^ surrounded by 
the above curves. We use the term Rij^k to denote the interior of the region 

Ri,j,k- 

The intersection between the region Rij^k and the partitioning curves is 
called a face and could be 0-dimensional, 1-dimensional, or 2-dimensional 
which are named as vertex, edge, and facet, respectively (Fig|3]). The set 
V{*) stands for the vertices that belong to * (* can be a facet, a region Rij^k, 
or the sphere Sr^), and F{v) is the set of facets that the vertex v belongs to 
them. Each region Rij^k has eight vertices. The vertices of Rij,k, V{Ri,j,k), 
are labeled as follows (FigJl]): 
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We have presented the vertex in the form of fm^,me,TOr) where the 
binary indices m<^, m^i, and show which partitioning curves that have 
generated the vertex Vm- For example, if = 1, it shows that the vertex 
v,n of the region touches the curve rj+i, and if rur — 0, it touches the 

curve Ti. 

The element Rij,k has six facets {F^^, F~, F^, F^, Ft, Ft} and correspond- 
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ingly, six outer normal vectors {n^ , n~, , n^, n^, n^} (Figj5]). The excep- 
tion is when the region Rij^k touches the origin or the z axis. In this case, 
some of the vertices are coincident. 

In sphere Sr^, let's define S as the sphere surface and E as the set 
of all of the edges and the vertices. Also, consider the detection element 
d{[i,j,k], [i',j',k']) = Rij^k n Ri'j'^k' — E, which is defined for two regions 
Ri j k and Ri'j'^k' that are adjacent in a common facet (the order is not im- 
portant). Indeed, the detection elements are the facets in which the edges 
and the vertices are excluded. With this procedure, the sphere Sr^ has been 
partitioned into E U Rij,k U d{[i, j, 

k], k'])US, where 1 < < Ur — l, 1 < < ng — 1, 1 < k, k' < — 1 
and S = S — E. Correspondingly, consider E, Rij^k, d{[i,j,k], [i',j',k']), S 
as the labels for these partitioning elements, where 53(f) = r relates the label 
f to the set r. This partitioned space can be captured by the equivalence 
relation Q = {(xi,X2)|3f G {E, Rij^k, d{[i, j, k],[i' , j' , k']), S} s.t. Xi,X2 G 
53(f) and 1 < < Ur — 1, 1 < < ng — 1, 1 < k,k' < — 1}. Cor- 
respondingly, the projection map t^q{x) shows the partitioning element that 
X belongs to it: t^q{x) = f G {E, Rij^k, d{[i, j, k],[i' , j' , k']), S} s.t. x G 
r and ^^(f) = r, where 1 < < n^ — 1, 1 < j,j' < ng — 1, 1 < k, k' < — 

In the next section, we will utilize the properties of multi-affine functions 
over the above partitioned space. 

4. MuIti-afRne vector fields over spherical partitioned space 

Multi-affine functions are a class of functions defined as follows: 
Definition 1. Affine function f2^] 
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A function g = {gi, g2, gm) '■ IR™' is said to be multi - affine, if all 

gi{xi,X2, ■■■,Xn) : M" — > M, i = 1, ...,m, can be affinely described with respect 

to each input parameter Xki meaning that if all input parameters are fixed 

I 

and only Xk changes, then for every set of ai,a2, ■■■,ak that ^ aj = 1: 
I I 

4.I. Properties of Multi- affine functions over the partitioned space 

The following proposition shows that a multi-affine function defined over 
the element Rij^k, can be uniquely expressed in terms of its values at the 
vertices of Rij^k- 

Theorem 1. For a multi - affine function g{x) : 5*^^,^ — ?■ M.^, the following 
property holds: 

Wx = {r,9,(f)) e Rij^k ■ gix) = ^\mg{vm), m = 0,2,...,7, (4) 

rra 

where Vm, m = 0, 7, are the vertices of the element Rij^k md Am can be 
obtained uniquely as follows: 

Am = A::^'-(1 - A,)i-'"^A7(l - Ae)^-'"''A7(1 - A^)l-"^^ (5) 

where rrir, mg, are corresponding binary digits of the index m as 
declared in ^ and 

Ti+l - Ti dj+i - Bj (pk+i - (pk 

proof: See the Appendix for the proof. ■ 

Remark 1. It can be verified that the resulting coefficients Am has the prop- 
erty that Am > and J2m -^m ~ m = 0, 1, 7. 

Theorem [1] also holds true for the points on the facets as described in the 
next proposition: 

Proposition 1. For a multi - affine function defined over Ri,j,k, for all of 
the facets F^ of Rij^k, Q € {r,0,(j)} and s G {+,—}, the following property 
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holds: 

Wx = {r,9,4>) e : g{x) = A^^?^) , G V{F^). (6) 

proof: It is a special case of Theorem 1 and, to prove this proposition, it just 
needs to follow only steps 1 and 2 in the proof of Theorem 1. ■ 

In the following proposition, we will show that in Theorem [H the coef- 
ficients are unique and there is one and only one multi-affine function over 
the sphere Sr^ that takes fixed values like g{vm) at the vertices of Rij^k- 

Proposition 2. Consider a map g : v{Sr^) — t- M^. Then, there exists one 
and only one multi-affine function f : Sfi^ — > defined over the region 
Ri,j,k with the vertices Vm, m = 0, 7, that satisfies fivm) = givm), for all 
of the vertices. 

Proof: See the Appendix for proof. ■ 

4-2. Utilizing the Multi-affine functions over the partitioned space 

For a multi-affine vector field defined over the region Rij^k, two important 
control features can be defined: the invariant region in which the system 



trajectories do not leave the region (Fig, 6(a)) and the exit facet through 
which the trajectories of the system leave the region (Fig 6(b)). The formal 
definition is given as follows: 



Definition 2. Invariant region 

In the sphere Sr^ and the multi - affine vector field x = g{x), g : Sr^ — M^, 
the region Rij^k is said to be an invariant region, if 



Wx{0) e Rij^k ^x{t) e Rij^k fort>0 



Definition 3. Exit facet 

In the sphere Sr^ and the multi - affine vector field x = g{x) , g : Sr^ — )■ M'^, 
the facet F^, q = {r, 9, ands = {+, — } is an exit facet, if Wx{0) G 
Ri,j,k =^ 3r (finite) > and 3e > satisfying: 

1. x{t) G Rij,k for t G [0,r) 

2. x{t) G fort = T 

3. x(t) ^ Rij^k for t e {r,T + e) 
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(a) (b) 
Figure 6: (a) Invariant region and (b) Exit facet. 

In the following theorems, we will find what conditions are required to 
construct a feedback controller to have an invariant region Rij^k, or an exit 
facet F^. 

Theorem 2. Sufficient condition for Rij^k to be an invariant re- 
gion: For a multi - affine vector field x = g{x), g : Sr^ — )■ M^, Ri,j,k is an 
invariant region if for each facet F^ and its corresponding outer normal n^, 
q E {r, 9, 0} and s E {+, — }; 

n',{y f.g{vj <OWvmE V{F^), Wy E F^ (7) 

proof: We should show that ng{y)^ .g{y) < holds true for any y E F^ and 
for all Fg. First, according to Proposition [H we know that: 

Vl/ E F^ : giy) = >^mg{v^) .VmEV{ F^) (8) 

Using this value of g{y), we can write: 

nl{yf-g{y) = n^yf. J2 ^mg{vm) = ^^m nl{yf.g{vm) (9) 

Now, according to the problem assumption described in (I7j), we know 
that nq{yY .g{vm) < for all Vm E V{F^) and y E F^. On the other hand, 
according to Remark [H we have Am > and 'Ylm -^m = 1- Hence, from ([9]), 
it can be concluded that n^^iji)^ -giy) < 0: 

This means that the trajectories of the system cannot leave Rij^k through 
the facet F^. Since this is through for all of the facets, the trajectories of the 
system will not leave the region Rij^k and will remain inside it forever. ■ 

Theorem 3. Sufficient condition for an Exit facet: For a multi - 
affine vector field x = g{x), g : Sr^ — t- M'^, the facet F^ with the outer 
normal n^^, q E {r, 9, 0} and s E {+, — }, is an exit facet if : 
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1. n^,{yY.g{v^) < \/v^e ViF^!), G F^! , q' ^ q, or s' ^ s 

2. nl{yY.g{vm) > Vt;„ G ^^(i^ij.fe), for all y G F^* 

proof: The first requirement guarantees that the trajectories of the system 
do not leave -Rij,^ through the non-exit facets F^, ^ F^. This has already 
been proven in Theorem [2J The second requirement is to drive the trajectory 
of the system out through the facet F^. Based on the assumption, for all 
y G F^ and for all Vm G V{Rij^k), we have: n^^y)^ .g{vm) > 0. According 
to Theorem [H for the multi-affine function g, there exist Am such that Vx G 
Ri,j,k ■ 9{x) = Y.m ^mgivm), m = 0, 1, 7. Slucc > and Y.m ^rn = 1, 
then n^qijiY .\m9ivm) > for all Vm- This will lead to have n^^^y)'^ .g{x) > 
for all X G Ri,j,k, which means that the trajectories of the system have a 
strictly positive velocity in the direction of n^ly) steering them to exit from 
Ri,j,k through the facet F^. ■ 

Here, we will present some of the properties of the properties of the exit 
facet controller that will be used through our further derivations. 

Lemma 1. For a multi-affine vector field x = g{x), g : Sr,^^ — )■ M'^, in a 
region Rij^k with the exit facet F^, constructed by Theorem\^ the following 
properties are concluded: 

1. The trajectories that leave the region do not return hack any more. 

2. The points on the exit facet are not reachable from other points on the 
facet. 

3. The trajectory that has reached the exit facet, leaves it immediately. 

proof: As observed in the proof of Theorem[3l respecting the second condition 
of this theorem leads to have n'^{y)'^ .g{x) > for any x G Rij,k and any 
y G F^. In particular this is true for the points on the facet F^. Hence, 
n^qiy)'^ .giy) > 0, G F^. This strictly positive inequality guarantees that 
the trajectory that reaches the exit facet, leaves it upon reaching the facet 
so that it can neither move along the facet nor return back. ■ 

Lemma 2. For a multi-affine vector field x = g{x), g : Sr^ M.^, in a 
region Rij^k with the exit facet F^ constructed by Theorem\^ the trajectories 
that leave the region only can pass through the detection elements. 

proof: As we saw in the proof of Theorem [21 for all points on the non-exit 
facets Fg, , we have n'^qiiyY .giy) < 0. This strictly negative inequality shows 
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that the trajectories of the system cannot pass through the non-exit facets 
including the edges and the vertices that belong to them. In particular, the 
trajectories cannot cross the edges and the vertices that are common between 
the non-exit facets and the exit facet. On the other hand, Theorem [3] shows 
that the trajectories of the system cannot remain inside the region. Hence, 
the only way is that the trajectories pass through the the internal area of the 
exit facet, which we have called it as the detection element. ■ 

Proposition 3. For a multi-affine vector field x = g{x), g : Sr^ — t- M^, in 

a region Rij,k with the exit facet constructed by Theorem\^ ally G F^\E 
are reachable from a point inside the region Rij^k- 

proof: Since any y G F^ is not reachable form the adjacent region (Part 1, 
Lemma d]) or from another point on F^ (Part 2, Lemma [T]), then, considering 
ng{y)^ .g{y) > 0, by continuity of g, there is a point inside the region Rij^k 
on the neighborhood of y from which y is reachable. ■ 



5. Control over the partitioned space 

5.1. Eligible sets for Invariant region 

Consider the multi-affine vector field x = g{x) = h{x,u{x)) over the 
partitioned space. Following from Theorem [2], to construct the region Rij^k 
as an invariant region, it is required to satisfy inequality ([7]) for all of the 
facets of the region. Starting from F+, we then need to have : 

nf. g{vm) < for G V{F+) (10) 

where: 

n+ = (1, e, 0) ej<e< and <Pk<<P< <Pk+i (11) 



Solving these inequalities is not an easy job. In [26| a geometric way is 
introduced to solve these inequalities in a polar coordinate system. To sat- 
isfy ffTOj) . extending this method to the spherical coordinates, the value of 
g{vm) should be chosen from a set gp+{Rij^k), for all Vm G \^(F+), where 
9F+iRi,j,k) = {(r, 9, 0)1^+1 + ^<9<9j + ^and ^k+i + f < < 0fc + f }■ 
As 9 and (p must be within certain ranges: < ^ < 27r, < < tt, we define 
the following functions: 
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and 



1 for 2k7i <a< {2k + 1)-k 

-1 for i2k + l)7r < «< {2k + 2)7r 



range{r, 6, 0) = (r, 6', 0'), where 

(e -2k7r for h{9)h{(p) > 0, 2A;7r < ^ < (2A; + 2)7i 

9' = le-{2k- 1)71 for h{e)h{(j)) < 0, 2A;7r < e < (2fc + ijvr 

[e- {2k + 1)tt for h{e)h{(t)) < 0, (2A; + l)7r < ^ < (2A; + 2)ti 

and 

- 2A;7r for 2k'K < (p < {2k + l)7r 

{2k + 2)Ti-(p for (2A; + l)7r < < {2k + 2)7r 

Using these functions, the ehgible facet sets for gp+ and other facets can 
be obtained as follows: 





= {range{r,e,(p)\9j+i + 


l<9 <9j + ^ and 


0fc+l 


+ f < < 0fe + f } 






= {range{r,e,(j))\9j+i - 


^<9 <9j + l and 


0fc+l 


- f < < 0fc + f } 




9f+ iRid,k) 


= {range{r,e,(j))\ej+i - 


TT <9 < 9j + i] 


9f- i^i,j,k) 


= {range{r,6,(j))\6j < 9 


< 9j + tt} 


9F+i^i,j,k) 


= {range{r,9,(l))\(j)k+i - 


- TT < < 0fc+i} 


9f- iRi,j,k) 
\ 4> 


= {range{r,9,4>)\4>k < q 


^ < 0fc + vr} 



(12) 



Now, since vq e V{F^)nV{Fg')nV{F^ ), to satisfy ©, the value of g{vo) 
should be chosen from the set Invo = gp- fl gp~ fl gp~ . The same procedure 
should be followed for other vertices. The eligible vertex sets for the region 
Ri^j^ki are as follows: 



14 



Invo{Ri,j,k) = {range{r, 6, (f)) \ 6j < 6 < 9j + | and (f)k < (f) < (f)k + -^j 
Invi{Rij^k) = {range{r, 9, (f)) \ 9j^i + j<9<9j + tt and 

(pk+i + f < < 0fc + tt} 
Inv2{Ri,j,k) = {range{r, 9, 0)| 9j+i - ^ <9 < and 0fc < < 0fc + f } 
Inv^i^Rij^k) = {rangeir, 9, 0)| 9j+i — vr < 6* < 6*^ + ^ and 

0fc+i + I < < 0fc + vr} 
< InVi{Rij^k) = {range{r, 9, (f))\9j < 9 < 9j + | anrf (f)k+i - f < < 0fc+i} 
Inv^{Rij^k) = {range{r, 9, 0)| ^^j+i + | < 6* < 6*^ + vr and 

0fc+l + TT < < 0fc + ^} 

InvQ^Rij^k) = {rangeir, 9, 0)| — | < 6* < 6'j_|_i and 

0fc+i - f < < 0fc+i} 
Inv7{Rij^k) = {range{r, 9, 0)| 9j^i + -K<9<9j + ^ and 
0fc+i + vr < < 0fc + ^} 

(13) 

5.2. Eligible sets for Exit facets 

Assume that in the region Rij^k, we are going to construct F~ as an exit 
facet. Following from Theorem [3l for all of the vertices, Vm, rn = 0, ...,7, 
two conditions should be checked. Let's start with vq. As Vq belongs to the 
facets F~ , F^", , and F~ is the exit facet, to satisfy the first condition of 
Theorem [3], it is sufficient to find g{vo) such that ng{x)'^.g{vo) < for all 
X G Fg" and n'^{x)'^ .g{vo) < for all x e F^ . To satisfy these inequalities, it 
needs to have g{vo) G gp- H gp-. 

To satisfy the second condition of Theorem [31 it needs to find g{vm), 
171 = 0, 7, such that they satisfy n~ {yY .g{vm) > for all y G F~. In this 
case, the value of g{vo) can be chosen from the set hp~^ = {g\n~{x)^ .g > 
V X G F-}. 

To satisfy both conditions, g{vQ) should be chosen from the eligible exit 
facet set Exo{F~ (Rij^)) = hp~ ^Qp- ^dpg- '^^^^ procedure can be similarly 
continued to find the eligible sets for other vertices, Exm{F~ {Rij±)) , m = 
0, 7. Following this procedure, to have F~ as an exit facet of the region 
Rij^k, the eligible exit facet sets are: 
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' Exo{F-{Rij^k)) = Exi{F~{Ri^,^k)) = 

{range{r, 6, 0)| 6j+i + ^<6<6j + TC and 0^+1 + f < < 0fc + tt} 
Ea;2(F-(i?,,,-fc)) = Ex3{F-{Ri^,^k)) = 

{range{r, 9, 0)| 6j+i + ^<9<9j + n and (pk+i + vr < < 0^ + ^} 

' EX4 (F- fc)) = EX5 (F- fc)) = 

{range{r, 9, 0)| 9j^i + 'K<9<9j + ^ and 4>k+i + f < < 0A: + ^r} 
Exe{F-{Ri^,,k)) = Ex7{F-{Ri^j^k)) = 

{rangeijr, 9, \ 9j + n < 9 < 9j + ^ and (pk+i + 7r<0<0fc + ^} 

(14) 

The procedure that makes other facets as exit facet is similar. 

5. 3. Construction of the controllers 

By proper selection of the control value it is possible to tune the 

multi-affine vector field, x = g{x) = h{x,u{x)), at the vertices, so that the 
region Rij^k becomes an invariant region or one of its facets becomes an exit 
facet. Indeed, to make the region Rij^k as an invariant region, the value of 
the control signal at the vertices, u{vm), should be chosen such that g{vm) = 
h{vm,u{vm)) falls in the set Um{Inv{Rij^k)) = InVm{Ri,j,k) n U, for m = 
0, 7, where InVm{Ri,j,k) is the eligible set for the vertex Vm as introduced in 
( !T3|) and U is the velocity bound, which comes from the practical limitations. 
If Um{Inv{Rij^k)) 7^ 0, for all m = 0, 7, then making the region Rij^k as 
an invariant region is feasible. Based on Theorem [H having the value of the 
control function u at the vertices Vm-, it is possible to construct the multi-affine 
controller u{x) for all x G Ri,j,k- Later, we will use the notation Co{Rij,k) to 
label this controller. In the case that for all of the regions Rij,k, the values 
of the vector field at the vertices, u{vm), are the same, then we can simply 
use the label Cq to denote the controller Co(-Rj,j,fc). 

To make the facet F^ as an exit facet, similar to the invariant controller, 
it is sufficient to choose the values of u{vm) such that g{vm) = h{vm,u{vm)) 
falls in the set Urr,{Ex{F^{Rij,k))) = Exrn{F'q{Rij,k)) n U. Therefore, for a 
region Ri^j^k, if all of Um{Ex{Fg (Rij^k))) 7^ 0, then corresponding to each of 
the exit facets, F+, F~ , Ffj~, F^", F^, F^, there are controllers that can 
make them as exit facets. We label these controllers as C^, C~, C^, Cg , 
C^^C^j^, respectively. 
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Remark 2. For a convex velocity constraint set, U, since g{x) is a multi- 
affine function, respecting the velocity bounds at the vertices, leads the system 
to respect these conditions for all x G Ri,j,k- 

6. Spherical abstraction of the state space 

Now, consider the original system which is defined over the partitioned 
space. The equivalence relation Q, defined in Section El describes this 
partitioned space. This system can be captured by the transition system 
Tq = {Xq, Xqo, Uq, ^q, Yq, Hq), where 

• Xq = EURij^k^d{[i, j, k], [i',j', k'])US is the set of system states, where 
l<i,i'<nr — l, l^j,j'^ng — l, l<k,k'<n^ — l. 

• Xqq is the set of initial states. Assuming that the system initially starts 
from inside one of the regions Rij^k, -^Qo = [J^i,j,k, where 1 < i,i' < 
Ur — I, 1 < < ng — 1, 1 < k, k' < n,j) — 1. 

• UQ = UaU Ud, where 

— Ua = {C:^, C~, C^, Cg , C^, , Cq} is the set of labels corre- 
sponding to the controllers that can make the region Rij^k as an 
invariant region or can make one of its facets as an exit facet. For 
these control labels, the sets of control actions that can be acti- 
vated in this region are : r{Cg) = {u{x)\u{x) = J2m^rnU{Vm), m = 

0,1,. ..,7, E V{Rij,k), uivm) e Urn{Ex{F^))}, and r(Co) = 

{u{x)\u{x) = Y.^mU{Vm), Vm ^ V{Rij^k), u{Vm) E U^{Inv{Ri^j^k))] ■ 

\m can be obtained by (E]). 

— Ud = Uc U Ue is the set of the detection events, where Uc = 
{d{[i,j,k], [i',f,k'])\ l<i,i' <nr-l, 1 < J,/ <ne-l, 1 < 
k,k' < — 1}. d{[i, j, k],[i' , j' , k']) is an event that shows the 
detection element d{[i,j,k], [i',j',k']) has been crossed and Ue is 
the set of external events such as entering into an alarm zone of 
collision. 

• {x, x\ v) €— >Q, denoted by x x', if and only if one of the following 
conditions holds true: 

1. Actuation: 
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- Exit facet: v G {C^\q E {r, 6^, 0} , s G {+,-}}; T^gix) 7^ 

'^gi^') ; ^Qi^) = Ri,j,k] T^Qix') = di[ij, k], k']); 3T{fimte) ande > 
s.t. ■ip{t) : [0, T+e] M? is the solution of x = h{x, r{v)); tp{0) = 
x; ipir) = x'; 'KQ{i){t)) = ttq{x) for t G [0,r), and 7TQ{ilj{t)) 7^ 
nqix) for t G [r, r + £:]. Here, r{v) is the continuous con- 
troller corresponding to the the control label v, which can be 
constructed as discussed in Section 15.31 

- Invariant region: v = Cq; itq{x) = ttq{x') = Ri,j,k; V'(^) : 
R"*" — )■ is the solution of i = h{x,r{v)); ip{0) = x; iP{t) = 
x', and TTQ^ipit)) = ttq{x) for all t > 0. 

2. detection: 

— Crossing a detection element: v G Ud, T^qix) 7^ 7rQ(x'); ngix) = 
d{[i,j,k],[i',j',k'])] 71q{x') = Ri'j'^k'] 30 < e < r and 3w G 

G {r,e,0}, s G {+,-}} s.t. ij{t) : [0,r] ^ R^ is the 
solution of i; = h{x,r{w)); i/j{e) = x; ipir) = x']7iQ{ip{t)) = 
Ri,j,k for t G {0,e), and 7CQ{^{t)) = Ri'j'^k' for t G (e,r]. 

— External events: v G Ue, and x = x'. In this case, x is the 
value of the system state at the time instant that the event 
V appears. The external event does not affect the system 
dynamics. 

• Yq = Xq is the output space. 

• Hq : X Yq is the output map. Here, we have chosen Hq{x) = 71q{x). 

Although Tq contains only important transitions that either cross the 
boundaries or remain inside the regions, still it has infinite states and the 



analysis of such a system might be difficult. Abstraction [SJJ, is the technique 
that can reduce the complexity and can lead to a finite state machine for 
which the DES supervisory control tools can be used for the system analysis 
and control synthesis. To do so, each partitioning element, can be considered 
one state in the abstracted model. Hence, the abstract model is a tuple 
= (X^, Xg„, U^, ^5, Y^, H^), where 

• X^ = {Rij,k\ 1 < i < Ur — 1, 1 < J < ""-e — 1, 1 < k < — 
l}[jW,J^k],[t',j',k'])[l < < n,. - 1, 1 < J,/ < - 1, 1 < 
/c, k' < — 1}, where Ri^j^k and c/([i, j, /c], j', k']) are the labels for 
the regions Rij^k and j, k], k']), respectively. Note that since 
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the system starts from a point inside the regions Rij^k and never crosses 
the edges or the vertices, the set E does not need to be considered in the 
abstracted system. Moreover, as the sphere Sr^ is the control horizon, 
its surface, S, should not be crossed. 

• X^o = {^ij-fcl 1 < < l<j,j'<ne-l, 1 < k, k' < n^-1}. 

• = UaU Ud is like what we have in Tq. 

• {r,r',v) G— T-^, denoted by r r', if 3v E U^, x E ^{r), x' E 3(r') 
such that X x'. 

• n = ^e- 

• H^{r) = r is the output map, which is selected as an identity map. 

In general, the abstract model contains all of the behaviors of the par- 
titioned system, however, the converse might not be always true. If the 
converse is also true, we say that they are bisimilar. A bisimulation relation 
between two transition systems can be formally defined as follows: 

Definition 4. [31] Given Ti={Qi, Q^, Ui, Yi, Hi), {i = 1,2), R is a bisim- 
ulation relation between Ti and T2, denoted by Ti ~ T2, iff': 

1. Vgi E Qi then 3q2 E Q2 that (gi,g2) e R and Vg2 G Q2 then 3qi E Q\ 
that (gi, ^2) G R- 

2. Vgi —7-1 q'l, and {qi,q2) G R then ^ Q2 such that q2 -^2 12 ^'^^ 
(?i)?2) ^ ^- ^Iso, Vg2 —^2 925 ^'^'^ {.11)12) G R then 3q[ E Qi such that 
qi -^i q[ and {q[,q2) E R. 

Theorem 4. The original partitioned system, Tq, and the abstract model, 
T^, are bisimilar. 

Proof: Consider the relation R = {{qQ,q^)\qQ E Xq, q^ E X^, andqq E 
^{q^)}. We will show that this relation is a bisimulation relation between Tq 
and Tg. 

To verify the first condition of bisimulation relation, for any gg E Xq^ 
there exists a region Ri^j^k such that qQ E Rij,k- For this region, there 
exists a label, R-ij^k such that Rij^k = ^{Ri,j,k) and Rij^k G X^^. Hence, 
(gg, Ri,j,k) G R. Conversely, it can be similarly shown that for any q^ E X^^, 
there exists a. qQ E Xg^ such that (g^, gg) E R. 
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For the second condition of bisimulation relation, following from the def- 
inition of T^, for any {qQ,q^) G R and Qq -^q q'q, there exists a transition 
q'^, where G ^{q'^), or equivalently {q'q.q'^) E R- For the converse 
case, assume that q^ q'^. According to the definition of i?, all x E ^{q^) 
are related to q^. Hence, to prove the second condition of the bisimulation 
relation, we should investigate it for all x e '^{q^)- Based on the control 
construction procedure, the labels u, q^, and q^ can be one of the following 
cases: 

1. u = Cq and q^ = q'^. In this case, since the controller Cq makes the 
region as an invariant region (Theorem^]), all of the trajectories starting 
from any qq G '^{q^) will remain inside the region '^{q^)- Therefore, 
for any qq G ^{q^), there exists a. q'q E ^{q^) such that qq q'q and 

2. u E Cg, q^ E {Ri,j,k\ I < i < rir — 1, I < j < ne — 1, 1 < A; < — 1}, 

and q'^E{d{[t, J, k],[t', f ,k'])\l<t,t' <nr-l, l<J,f<ne-l, 1< 
k, k' < n^ — 1}. In this case, based on Theorem[3]and Lemma|2l starting 
from any qq E '^{qs^), the controller C| drives the system trajectory 
towards the detection element '^{q'^)- Therefore, for any qq E Q'(g^), 
there exists a q'q E such that qq Ag q'q and q'q E ^{q'^- 

?>. uEUc = {d{\i,j,kl[i',j',k'])\l<i,i' <nr-l, 1 < J,/ < ne-l, 1< 
/c, /c' < — 1} and G {-Ri'j',A;'| 1 < < TT-r — 1, 1 < j' < Ji-e — 1, 1< 
k' < — 1}, and q^ E {c/([z, j, fc], [^', j', k'])\ 1 < i,i' < rir ~ I, 1 < 
j,j' < ng — 1, 1 < k, k' < n^f, — 1}. In this case, based on Proposition 
131 for any qq E Q'(gg) there is a controller v E that has leads 
the trajectory of the system from the region Rij^k to the point qq on 
the detection element d{[i, j, k],[i' , j' , k']). Since Ri'j'^k' is the unique 
adjacent region of the element Rij^k common in the detection element 
d{[i, j, k],[i' , j' , k']), based on the definition of the controller for the 
exit facet and Theorem [3l the controller v leads the trajectory of the 
system to a point inside the region Ri'j'^k' so that the detection event 
u = d{[i, j, k],[i' , j' , k']) is generated. Therefore, for any qq E Q'(g^), 
there exists a q'q E ^{q'^) such that qq Ag q'q and q'q E '^{q'^)- 

4. u E Ue is the external event. In this case, the state of system does not 
change, meaning that qq = q'q and q^ E q'^. Therefore, trivially for any 

Qq e ^{q^) and q^ g^, we have qq Ag q'q, where q'q E ^{q'^). 
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In all of the above mentioned cases, the second condition of bisimulation 
relation for the converse case holds true. Hence, and Tq are bisimilar. ■ 



7. Adopting the DES supervisory control to the abstracted model 

Following the above procedure, the partitioned system Tq was bisimilarly 
abstracted to a finite state machine T^. The advantage of this method is that 
the control synthesis of this finite state machine can be effectively handled 
within Discrete Event Systems (DES) supervisory control theory initiated by 



Ramadge and Wonham [22 



7.1. DES model of the plant 

The finite state machine can be formally presented by an automaton 
G = (X, E, a, Xq, Xm), where X = X^ is the set of states; Xq = X^^ C X is 
the set of initial states; X^ = {Rij,k\ ^ ^ j ^ — 1, 1 < k < — 1} is the 
set of final (marked) states; S is the (finite) set of events. The sequence of 
these events forms a string. We use e to denote an empty string, while S* is as 
the set of all possible strings over the set S including e. The language of the 
automaton G, denoted by L{G), is the set of all strings that can be generated 
by G starting from the initial states. The marked language, L^iG), is the 
set of strings that belong to L{G) and end with the marked states. L(G(xo)) 
is the set of strings that belong to L{G) and start from the initial state xq. 
L is the set of all prefixes to the strings that belong to the language L. 

Here, the event set S consists of the actuation events Ua = {Cq\q G 

{r, 6',0}, s G {+, —}}U{Co}, the crossing the detection events f/c = {d{\iii,k], 
j',fc'])|l < < Hr - I, 1 < < - 1, 1 < k,k' < - 1}, and 
the external events Ue- Also, the set f/g only contains the event ca for the 
collision-alarm; Indeed, the event ca, will be generated when the system 
detects that another agent is located on the way of the follower towards 
the desired position. The event set E consists of the controllable event set 
= Ua and uncontrollable event set E^c = Ud = Uc U Ue- The uncon- 
trollable events are those that cannot be affected by the supervisor. In au- 
tomaton G, a : X X E — )■ X is the transition function, which is a partial 
function and determines the possible transitions in the system caused by an 
event. This function is corresponding to — in T^, so that for any r A-^ r' we 
have a{r,v) = r'. Based on the definition of and constructed controllers 
Co, C+, G-, C+, C+, C+, G7, we have: 
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Some parts of the graph representation of the system automaton are 
shown in Fig. [71 In this automaton, the arrows starting from one state 
and ending to another state represent the transitions, labeled by the events 
belong to E. The entering arrows stand for the initial states. As it is shown 
in Fig. [71 the system could start from any of the states Rij^k- 




Figure 7: DES model of the plant. 



In this model, when the system is in the state Ri,j,k, the command C~ 
leads the system to an unknown region and the system would become non- 
deterministic. Hence, as it is reflected in the expression of a, the command 
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cannot be activated in this region. Similarly, the commands and 
cannot be activated in Rij^n^-i and Rij,i, respectively. Moreover, in the 
region Rn^^ij^k the command cannot be activated as it leads the system 
outwards the control horizon. These restrictions are already considered in 
the definition of a. 

Now, the aim is to design a controller to bring the trajectories to the final 
states regardless of the initial state of the system. The control design 

issues are discussed in the next section. Before that, to simplify the DES 
model of the plant and to reduce its number of states, we can merge the states 
that have a similar situation in terms of the control labels that can be acti- 
vated in these states. In this case, the refined model (Fig. |8]) and the original 
DES model (Fig. [7]) are language equivalent, meaning that L{G) = L{Gref) 
and Lm{G) = Lm{Gref) [32"]. To do the refinement, first, all of the detection 
states {d{[i,j,k], [i'J\k'])\ 1 < < ra^- 1, 1 < jj' < ne - I, l<k,k'< 
— 1} can be merged into a state called D. Then, one can merge similar 
states based on the events that can be activated for each region. Indeed, as 
described in the definition of a, for the states that are adjacent to the sphere's 
boundary, the event cannot be activated. For the regions that touch the 
positive side of z axis, negative side of z axis, and the origin of the sphere, the 
events C^, C^, and C~ cannot be activated, respectively. Following these 
rules, the states that are adjacent to the boundary of the control horizon 
and does not touch the z axis, {Rnr-i,j,k\^ ^3 ^''^e — 1, 1 < k <n^ — 1}, 
can be aggregated to the state shown in Fig. [HI Other merged states are: 
A^i = {-R„,-ij,i|l < i < ne - I] , Nn = {-R„,,_i,j>^^_i|l < j < rie - 1}, 
R = < j <ng -1, 1 < k < Ri = {^ij, i|l < J < ng - 1}, 

Rn = {-Rlj>^_l|l < j < ng - 1}, P = {Ri,j,k\l < i < nr - 1, 1 < j < 

ng — 1 , 1 < k < — 1} , Pi = {Rij^i\l < i < nr — I, 1 < j < ng — 1}, and 
Pn = {-Rij,n^_i|l < i < Ur — 1, 1 < j < ng — 1}. The hst of the events that 
can be activated in each merged state, and fire a transition to the detection 
state D form the set 7c(*), which is defined as follows: 
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7c(r) 



' n^x 

X^r 1 '^r I'^e I'^e i^(t> i^(t> 5 

sn+ n- n+ n~ n^x 
r^^i 



r = P 
r = Pi 

r = Pn 
r = N 
r = Ni 
r = Nn 
r = R 
r = Ri 
r = R„ 



(15) 



Transition from the detection state D to one of the merged states gen- 
erates the detection events that are presented by the set 7d(*). For in- 
stance 7d(P) = {d{[i,j,k],[i',i',k'])\l < i < - 1, 1 < < uq - 
1, I < k < - I, I < i' < rir - I, I < k' < - 1} and 7d(Pi) = 
{d([z,j,/c],[2',/,l])|l<2<n,-l, l<j,/<ne-l, l<A;<n<^-l, K 
i' < rir — 1}. 




Figure 8: Refined DES model of tlie plant. 
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1.2. Design of the supervisor 

The logical behavior of the system can be modified by a discrete supervi- 
sor to achieve a desired order of events. Indeed, the supervisor, S, observes 
the executed events of the plant G and disables the undesirable controllable 
strings. Here, we assume that all of the events are observable. The language 
and marked language of the closed-loop system, L{S/G) and Lm{S/G), can 
be constructed as follows: 

(1) £ e L{S/G) 

(2) [{s e L{S/G)) and {sa E L{G)) and {a G L{S))] ^ {sa e L{S/G)) 

(3) L^{S/G) = L{S/G)fM^{G) 

where s is the string that has been generated so far, and a is an event, 
which the supervisor should decide whether keep it active or not. 

Within this framework we can use parallel composition to facilitate the 
control synthesis. Parallel composition is a binary operation between two au- 
tomata. Here, the parallel composition is used to combine the plant discrete 
model and the supervisor. 

Definition 5. (Parallel Composition) [35] Given G = {Xq, Sg, «G) xqq, Xma) 
and S = {Xs, S5, as, ^Os, ^ms)j ^ci = G^S = {X^, S^/, ad, Xq^,, -^m^J 
is said to be the parallel composition of G and S with X^ = Xq x Xs, 
= U Eg, xo^; = (xocXog), Xm^i^ = Xmc X X^s, o-f^d Vx = (Xi,X2) G 
Xcuo e T^d, then ad{x,a) = 

• {aG{xi,a),as{x2,(T)) 

if adxi, a)\ and as{x2, cry. and a E Sg* H ^5 
< •{acixi, (x), X2) if acixi, a)\ and a G - 

• (xi, as{x2, cr)) if as{x2, a)\ and a G S5 - Eg 
•undefined otherwise 

where a*(x, cr)! shows the existence of a transition from the state x by the 
event a in system *. In this definition, the initial condition of the automata 
was assumed to be singular. Extending this definition for the case that 
automata G and S have the initial sets Xq^ and Xq^, the initial set of the 
composed system will be Xq^^ = T{Xo^,Xo^) C Xq^xXq^, where the relation 
T describes the initial states in G and S that are coupled to synchronously 
generate a string in the composed system. 
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In fact, parallel composition synchronizes operand systems on their com- 
ment events, however, their private events can transit independently. Next 
lemma and Corollary use the parallel composition to obtain the closed-loop 
system. 

Lemma 3. fs^] Let G = {X, S, a, Xq, X^), be the plant automaton with 
the initial state of Xq and K C J]* be a desired language. There exists a 
nonblocking supervisor S such that L{S/G) = L{S\\G) = K if and only if 
^ ^ K = K <Z L{G) and K is controllable. In this case, S could be any 
automaton with L{S) = Lm{S) = K . 

Using the above lemma and following the definition of the parallel com- 
position, the result can be extended to a plant with several initial states: 

Corollary 1. Let G = {X, E, a, Xq, X^) be the plant with the initial state 
set Xq = {xq,Xq, ...} and K = [J Ki C T,* be a desired language, where Ki is 
the desired language that should be generated starting from Xq. If ^ ^ Ki = 
Ki C L{G{xq)) and Ki is controllable for all i = 1,2, \Xo\, there exists a 
nonblocking supervisor S such that L{S/G) = L{S\\G) = K. In this case, 
S could be any automaton that has the initial state set Sq = {sg, Sq, s^}, 
m < \xq\, and for any Xq there exists a Sq, (xq, Sq) G T, which satisfies 
Lm{S{sl)) = L(5'(sq)) = Ki, where T is the coupling relation between the 
initial states of supervisor S and the plant G. 

Now, using the parallel composition and the above corollary, we will de- 
sign the supervisor for reaching the formation, keeping the formation, and 
collision avoidance modularly. 

7.2.1. Design of the controller for reaching and keeping the formation 

For reaching the formation, it is sufficient to drive the follower UAV 
directly towards one of the regions Ri,j,k, 1 <j ne — 1, 1 < k < Ufj, — 1, 
located in the first sphere. After reaching Rij,k, the UAV should remain 
inside it, forever. This specification, Kp, is realized in Fig. O In this 
figure, the initial state Pf is considered to be coupled with the states P, PI, 
Pn, N, Ni, Nn, in Fig. ID Being in one of these states, the UAV is not 
in the first sphere, and the event G~ will be generated to push the UAV 
towards the origin. Entering into a new state, the event d{[i,j,k], [i',j',k']) 
will appear to show the current state of the system. This will continue until 
the event d{[i,j, k], k']) be generated, which shows that the formation is 
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reached. In this case, the event Cq will be activated, which keeps the system 
trajectory in the first region. Since, there is another module to handle the 
collision avoidance, the formation supervisor does not change the generatable 
language after the event ca, and lets the collision avoidance supervisor disable 
undesirable events as it will be explained in the next section. It can be seen 
that Kp is controllable as it does not disable any uncontrollable event. 



ca 




Figure 9: The realization of the reaching and keeping the formation specification. 

Based on Corollary [H there exists a supervisor that can control the plant 
to achieve this specification. The supervisor is the realization of the above 
specification in which all states are marked. Marking all states of the super- 
visor allows the closed-loop marked states be solely determined by the plant 
marked states. The supervisor for reaching the formation and keeping the for- 
mation is denoted by 5*^. The closed-loop system can be obtained using the 
parallel composition: Gd = Sp/G = Sf\\G. Here, the coupling relation is as 
follows: T = {{Ru Rj), {R, Rj), Rf), (Pi, Pf), (P, Pj), (P„, P;), {N^, Pj), ( 
N, Pf), {Nn, P/)}- All of the events are common between the plant and the 
supervisor. Moreover, it can be seen that L{Sf) C L{Gref)- Therefore, 
knowing that L{G) = L{Gref), it will be concluded that: 

L{Sf/G) = L{G\ \Sf) = L{G) n L{Sf) = 

L{Gref) n L{Sf) = L{Sf) = Kf (16) 



Remark 3. If for any reason the follower UAV deviates from the required 
position and for instance, enters into one of the adjacent regions nondeter- 
ministically , the event d{\i, i,k],[i' , j' ,k']) will he generated, which informs 
that the UAV is entered into the region Ri'j\k'- Having this information. 
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the controller can resume its job from the newly entered region and push the 
UAV towards the origin. 

7.2.2. Collision Avoidance Supervisor 

When the follower UAV is going to reach the desired position, in some 
situations the follower may collide with the leader. More precisely, when the 
follower UAV is in the region Rij^k and the leader UAV is located in the 
region Ri'j^k, and i' < i, then the collision alarm, ca, will be generated. If 
we look at this problem from the relative frame point of view, the leader 
UAV has a fixed position in this frame. Therefore, when the follower detects 
that the leader is located on its path towards the desired position (center 
of sphere), it suffices that the follower turns to change its azimuth angle, 
9, by activating the event Cg. When, it enters into the region i?j j+i fc, the 
event d{[i,j, k], [i,j + 1, k]) G Ud will appear, and the colhsion alarm will be 
removed. In this case, the collision avoidance supervisor lets the formation 
supervisor resume the reaching the formation. To do so, the collision avoid- 
ance supervisor only changes the generatable language after happening the 
event ca and lets the rest be treated by the formation supervisor. The colli- 
sion avoidance supervisor, Sc, is shown in Fig. [TDi Here, the coupling rela- 
tion is T = {(Pi, Sc), (P, Sc), (P„, Sc), (N,, Sc), (N, Sc), (Ar„, Sc)}. Again, 
all the events are common between the plant and the supervisor. Hence, 
LiSc) C L{Gref) and L{G) = L{Gref) lead to: 

L{Sc/G) = L{G\\Sc) = HG) n L{Sc) 

= L{Gref) n LiSc) = L{Sc) = Kc (17) 

where Kc is the collision avoidance specification. 




Figure 10: Collision avoidance supervisor, Sc- 
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1.2.3. The closed-loop system 

For prefix closed languages Kp and Kc, we can apply modular synthesis 



33[, using composition of the plant, the reaching and keeping the formation, 



and the collision avoidance supervisor: 



Gel — G\\Sf\\Sc 
Here, the closed-loop language can be achieved as follows: 



(18) 



L{G\\Sf\\Sc) = L{G) n L{Sf) n L{Sc) = 
L{Sf) n L{Sc) = KFnKc 

The refined closed-loop automaton, G^i^^j, is shown in Fig. [TTl 



(19) 



c: 



D 



ca 



ca 



ca 



// o\ \r 




-^^ ca 










ca 






ca 



Figure 11: The closed- loop system. 



8. Construction of the hybrid controller 



To describe the partitioned system and its relation with the discrete su- 
pervisor, analogous with [l^ and referring to the definition of Tq, we can 
define an interface layer (Fig. 12(a)), which connects the supervisor to the 
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plant. The interface layer has two main blocks: the detector and the actuator. 
The detector converts the continuous time signals to a sequence of symbols. 
Upon crossing the detection elements, a plant symbol, d{[i,j, k], k']), is 
generated, which informs the supervisor about the current situation of the 
plant. Based on the observed plant symbols, the supervisor decides, which 
control signal should be given to the plant to satisfy the desired specification. 
This command has a discrete nature, but the control commands to be given 
to the plant need to be continuous. The actuator translates the discrete 
commands to the continuous ones: 



r{v) = u{x) = S„A„(x)M(t;m) (20) 

where v is the discrete command and u{vm) should be chosen from the sets 
Um{Inv{Ri j^iS)) V = Cq. Otherwise it should be selected from the set 
Um{Ex{Fg{Rij^k))) for v = C^. The coefficients Am(x), m = 0, 7, can be 
obtained from ([5]). The whole structure, the interface layer and the supervi- 
sor, is implemented on the follower UAV. 

Indeed, the plant and the interface layer elements, together with the 
actuator and the detector, form the transition system Tq. It was shown 
that this transition system can be bisimilarly abstracted to the finite state 



machine (Fig. 12(b)) for which we designed the discrete supervisor. Due 
to the bisimilarity of Tq and T^, the designed supervisor for can work for 
Tq so that the closed-loop system behavior does not change, leading to: 

Theorem 5. With the aid of interface layer, the discrete supervisor S = 
Sc\\Sf can be applied to the original partitioned system, Tq, so that the 
closed-loop system satisfies the required specification, Kp n Kc- 



9. Hardware-in-the-Ioop simulation results for the formation algo- 
rithm 

To verify the proposed algorithm we have used a hardware-in-the-loop 



simulation platform [3J| developed for NUS UAV helicopters [27|. In this 
platform, the nonlinear dynamics of the UAV has been replaced with its 
nonlinear model of the UAV, and all software and hardware components 
that are involved in a real fiight test, remain active during the simulation. 
In this platform, the control commands and references will be sent to the 
actuators, but the helicopter does not move as the motor is off. Indeed, the 
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Figure 12: (a) Interface layer structure and (b) Abstract model for the plant and interface 
layer. 



variables of the nonlinear model will change, and instead of the measured 
values, these values will be given back to the controller to compute new 
control values. Consequently, the simulation results of this simulator is very 
close to the actual flight tests. The controller is embedded in a PC\ 104 and 



the controller details and program code is explained in [35 



Now, consider two UAVs each of which are simulated by the hardware- 
in-the-loop software. The leader follows a predefined path and the follower 
receives the leader's data via a wireless communication board (IM-500X008, 
FreeWave). This multi-UAV simulator test bed is used to simulate the algo- 
rithm in two cases: 

1. Scenario 1: To monitor reaching the formation behavior of the UAV, 
we have used a fixed leader and the follower should reach the desired 
position. 

2. Scenario 2: To monitor how the follower is able to maintain the achieved 
formation, the leader tracks a circle path, and the follower should reach 
and keep the formation. 

9.1. Simulation of reaching the formation and collision avoidance 

The system dynamics, i; = m, is a multi-affine function. The controller, 
u{x), drives the UAV inside a spherical partitioned space. The control signal 
is generated using the control mechanism described in Sectional The control 
horizon is a sphere of diameter 50m. The partitioning parameters are selected 
as Ur = 15 and ne = 20 and = 10. To construct the controllers Co, C^, 
C~, C^, Cg , C^, and C^, we can apply Theorem [2] and [3l respectively. For 
example, to construct the controllers Cq and C~ , it is sufficient to choose the 
values of u{vm) from the sets Um{Inv{Rij^k)) and Um{Ex{F~ {Ri j^k))) given 
in f lT^ and ffH]) . respectively. 
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To simulate reaching the formation stage, assume that the leader has a 
fixed position and the follower should reach a desired position that has a 
certain distance from the leader. Also assume that Rm < d so that collision 
would not be happen. Considering that the follower has a relative distance 
{dx, dy, dz) = (17, 18, 8) with respect to the desired position, reaching the 
formation stage behavior is shown in Figure [121 The projection of the UAV 
position onto the x — y plane is shown in Fig. 14(a) The follower UAV's state 



variables are shown in Fig. 15(a) Moreover, the regions that the follower 
UAV has crossed are presented in Fig. 14(b) , where i, j, k are the indices of 
the traversed regions Rij,k- As it can be seen, the UAV has finally reached 



the final state -Ri,i3,5. The control signals are shown in Fig. 15(b) 




Figure 13: Reaching the formation stage. 



Now, with the similar situation as above, assume that the relative distance 
between the follower and the desired position is {dx, dy, dz) = (—17, —18, —8). 
With the partitioning parameters Rm = 50m, rir = 15, uq = 20, and = 10, 
the initial state of the system is -Rs.is.s- Assume that the leader is located 
in -R7,i3,5. Since the leader is located on the path of follower towards the 
desired position, a collision avoidance alarm will be generated to activate 
the collision avoidance mechanism. The collision avoidance behavior of the 
system is shown in Fig. [131 The projection of the UAV position onto the 
x — y plane is shown in Fig. 16(b), The state variables of the system are 
shown in Fig. 17(b) , The traversed regions are depicted in Fig. 17(a) , where 
i, j, k are the indices of the crossed regions. The follower, first has moved 
towards the region Rg 14 5 to avoid the collision, then it has resumed reaching 
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Figure 14: (a) Reaching the formation stage projected onto the x-y plane and (b) The 
traversed regions' indices in reaching the formation stage. 



Follower's state variables 



Control Signals 





(a) 



(b) 



Figure 15: (a) The state variables of the follower UAV in reaching the formation stage 
and (b) Control input signals in reaching the formation stage. 
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(a) (b) 

Figure 16: (a) The position of the follower UAV for the collision avoidance mechanism, (b) 
Relative distance of the follower from the desired position, projected onto the x-y plane. 



the formation to complete the mission. 

9.2. Simulation of keeping the formation 

To monitor reaching and keeping the formation, let the leader to track a 
circle with the diameter of 20m and with the altitude of 20m. Consider the 
partitioning parameters as the above scenario. Here, the follower is initially 
located at {dx,dy,dz) = (—20, —20, —20) with respect to the leader. It 
is expected that after a while, the follower reaches the relative distance of 
{dx, dy, dz) = (5, 5, 5) with respect to the leader. The behavior of the follower 



UAV is shown in Fig. 18(a) The follower UAV's state variables are shown 



in Fig. 18(b) As it can be seen the follower has finally reached the desired 



formation and has successfully kept it. 

9. 3. Simulation of a multi-follower scenario 

The algorithm can be extended to a multi-follower case. Each of the 
followers has his own supervisor, which makes it able to reach the formation, 
keep the formation, and avoid the collision. Each follower just needs to have 
the position and velocity information of the leader and also the position of 
the neighbor agents. In this case, the reaching the formation and keeping 
the formation is similar to what we designed so far, but a more complicated 
collision avoidance is required. We will consider the multi-follower case in 
our future research works and here, we just show how the idea can be used 
for a simple multi-follower case. 
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Follower's state variables 
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(a) 



(b) 



Figure 17: (a) The traversed regions' indices for the colhsion avoidance mechanism and 
(b) The state variables of the fohower UAV during the cohision avoidance mechanism. 





Figure 18: (a) The position of the UAVs in a circle formation mission and (b) The follower 
UAV's state variables during a circle formation mission. 
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Here,the partitioning parameters are Rm = 100m, rir = 20, uq = 5. 
Followers 1 is initially located at a point, which has a relative distance of 
{dx,dy,dz) = (—30.1,12.1,-12.3) with respect to the leader, while the de- 
sired relative distance is {dx,dy,dz) = (-15,-5,10). Follower 2 is initially 
located at a point which has a relative distance of {dx, dy, dz) = (3.8, 33.4, 0) 
with respect to the leader. The desired relative distance between this UAV 
and the leader is {dx,dy,dx) = (—15,-5,10). The position of the UAVs 
in the 3-dimensional space is shown in Fig. [19] and the projection of their 
position onto the x-y plane is shown in Fig. [201 As it can be seen, each UAV 
has reached the formation and has kept it successfully. 



80^ 






60-, 






40-, 
20-, 


Follower 2 reaches the formatioti 










0-. 








-20 i. 






F 



Follower 1 

Leader 

Follower 2 















iHi ttiH ti.imidtii.if 



3D 20 10 



T^^-20 "-30 "-40 --50 -60 -70 
V(m) 



Figure 19: Multi-follower formation control. 



10. Conclusion 

In this paper, a hybrid supervisory control was proposed for the three- 
dimensional leader-follower formation control of unmanned helicopters. The 
approach was based on spherical abstraction of the state space and utilizing 
the properties of multi-affine functions over the partitioned space. The finite 
state DES model of the plant was achieved by the proposed abstraction pro- 
cedure, and then, a discrete supervisor was modularly designed to satisfy the 
desired specifications. The designed supervisor is able to form the formation, 
maintain the achieved formation, and take care of the inter-collision between 
the agents. To link the discrete supervisor and the continuous model of the 
plant, an interface layer was embedded. Due to the bisimulation relation 
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Figure 20: The projection of the UAVs' position onto the x-y plane in mult i- follower 
scenario. 



between the partitioned plant and its DES model, it is guaranteed that the 
DES model and the original hybrid model of the plant exhibit the same be- 
havior. The algorithm was verified by a hardware-in-loop simulator and the 
presented results demonstrate the effectiveness of the designed controller. 

11. APPENDIX 

11.1. Proof for Theoremll\ 

let X = (r, 6, (p) G Ri.j^k- Therefore, from the partitioning procedure: 
r, < r < Ti+i, Oj < 9 < 9j+i, and (pk ^ 4> ^ 4>k+i- Hence, r, 9, and could 
be written affinely as: 

r = (1 — \r)ri + A^rj+i < Ar < 1 ^ Ar 
= (1 - \e)9, + \e9j+^ < A^ < 1 ^ A^ 
= (1 - A0)0fc + A^^fc+i < A^ < 1 ^ A,/, 

Consider a trajectory from the vertex vq = vqoo towards the point x only 

moving along the spherical axis. As an example, vq = {ri,9j,(f)k) — ^ xi = 

{r,9j,(f)k) — ^ X2 = {r,9,(j)k) xz = x = {r,9,(j)). In fact, in each step, 
we have changed only one parameter, and fixed the others to take the ad- 
vantages of the multi-affine functions. When in the function g{r,9,(j)), the 



ri+i-Vi 
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parameter 6 and are fixed and only r is varying, we use the notation gg^ to 
highlight the fixedness of 6 and 0. Similarly, we can define Qre and gre/,- As 
5^ is a multi-affine function, gg^, gre, and gr,f, are affine. Since in stepl, the 
parameters 6 and (p are fixed and only r is changing: 

9{xi) = gec^\e„4>dr) = ^e</.|e„0fe((l - ^r)ri + Xr^+i) = (1 - Xr)9ec^\e„<p,{ri) + 
Kge<i,\e„(i>k{ri+i) = (1 - K)g{ri,Oj,(j)k) + Xrg{ri+i,Oj,(f)k)- 
In step 2, the parameter 6 changes and the other parameters are fixed. There- 
fore: 

^(^2) = gr^{0)\r,c^^ = ^r</,|r,0fc((l - Xe)0j + XeOj+l) = (1 - Xe)grc^\r,c^,{Oj) + 

X9gr<p\r,^ki^j+i) = {l-Xe)g{r,9j,(f)k) + X0g{r,ej+i,(f)k) = {1 - Xe)g9^\e„^k{r) + 
Xege<p\ej+i,<Pki''^)- 

In step one, we have already obtained ge<j>\d ■ ,(i>k{r) . With the same procedure, 
we can obtain ge4}\9j+i,(t}k (r). Applying these two values in step 2, g{x2) will be: 
g{x2) = {1-Xg)[{l-Xr)g{ri, 9j, (t>k)+Xrg{ri+i, 9j, (j)k)]+X9[{l-Xr)g{ri, 9j+i, (f)k)+ 
Xrg{ri+i, 9j+i, = (1 - A0)(1 - Xr)g{ri, 9j, 0fc) + (1 - Xg)Xrg{ri+i, 9j, (pk) + 
Xeil - Xr)g{ri, 9j+i, (pk) + XeXrgiri+i, 9j+i, (pk) 

In Step 3, the parameter changes and the other parameters are fixed. 
Therefore: g{x^) = gre{(p)\r,e = (1 - X^)g{r, 9, (f)k) + X^g{r, 9, (pk+i) 

where g{r,9,(j)k) is obtained in Step 2 and g{r,9,(l)k+i) can be obtained 
in a similar way. Substituting these values in 5^(0:3) and rearranging the 
coefficients, the result will be as (jl]) and ([5]). 

11.2. Proof for Proposition [H 

The existence already guaranteed by Theorem [TJ For the proof of unique- 
ness, assume by contradiction that / is not unique, and there is another multi- 
affine function /' that f{vm) = fivm) = g{vm), or equivalently, /"(fm) = 
f{vm) — f'{vm) = 0. It could be easily proven, that if / and /' are multi- 
affine, f" = f — f also is multi-affine. By Theorem [H Wx G Rij^k '■ f'i^) = 
Yl Xmf"{vm) = 0, m = 0, 2, 7. Therefore, Vx G Ri,j,k, f{x) = f'{x). 
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